Search results for " Gesture"
showing 10 items of 44 documents
Body Gestures and Spoken Sentences: A Novel Approach for Revealing User’s Emotions
2017
In the last decade, there has been a growing interest in emotion analysis research, which has been applied in several areas of computer science. Many authors have con- tributed to the development of emotion recognition algorithms, considering textual or non verbal data as input, such as facial expressions, gestures or, in the case of multi-modal emotion recognition, a combination of them. In this paper, we describe a method to detect emotions from gestures using the skeletal data obtained from Kinect-like devices as input, as well as a textual description of their meaning. The experimental results show that the correlation existing between body movements and spoken user sentence(s) can be u…
Introduction to Gestural Similarity in Music. An Application of Category Theory to the Orchestra
2019
Mathematics, and more generally computational sciences, intervene in several aspects of music. Mathematics describes the acoustics of the sounds giving formal tools to physics, and the matter of music itself in terms of compositional structures and strategies. Mathematics can also be applied to the entire making of music, from the score to the performance, connecting compositional structures to acoustical reality of sounds. Moreover, the precise concept of gesture has a decisive role in understanding musical performance. In this paper, we apply some concepts of category theory to compare gestures of orchestral musicians, and to investigate the relationship between orchestra and conductor, a…
The facilitative effect of gestures on the neural processing of semantic complexity in a continuous narrative
2019
© 2019 Elsevier Inc. Gestures are elemental components of social communication and aid comprehension of verbal messages; however, little is known about the potential role of gestures in facilitating processing of semantic complexity in an ecologically valid setting. The goal of this study was to investigate whether cognitive load, as indexed by semantic complexity, is modulated by the presentation of gestures accompanying speech. Twenty healthy participants watched 16 video clips of a short narrative while instructed to carefully listen to and watch the narrator while functional magnetic resonance imaging (fMRI) data were acquired. The videos contained passages with and without various co-s…
Implicit visual analysis in handedness recognition.
1998
In the present study, we addressed the problem of whether hand representations, derived from the control of hand gesture, are used in handedness recognition. Pictures of hands and fingers, assuming either common or uncommon postures, were presented to right-handed subjects, who were required to judge their handedness. In agreement with previous results (Parsons, 1987, 1994; Gentilucci, Daprati, & Gangitano, 1998), subjects recognized handedness through mental movement of their own hand in order to match the posture of the presented hand. This was proved by a control experiment of physical matching. The new finding was that presentation of common finger postures affected responses differ…
Sonic Strokes and Musical Gestures - The Difference between Musical Affect and Musical Emotion
2009
Musicological theories traditionally address the expressive powers of music within a framework of signification, in which musical affect is considered as a meaningful musical gesture. Consequently, affect is regarded as a property of the music instead of as a bodily reaction of the listener. I will outline a different conception of musical affect that takes into account the bodily nature of perception. I will introduce the notion of sonic stroke, i.e. a sound that induces affect in the listener. A sonic stroke thus is responsible for the occurrence of musical affect. By relating the notion of sonic stroke to musical gesture the differences between the two can be articulated, as well as how …
A musical reading of a contemporary installation and back: mathematical investigations of patterns in Qwalala
2021
Mathematical music theory helps us investigate musical compositions in mathematical terms. Some hints can be extended towards the visual arts. Mathematical approaches can also help formalize a "translation" from the visual domain to the auditory one and vice versa. Thus, a visual artwork can be mathematically investigated, then translated into music. The final, refined musical rendition can be compared to the initial visual idea. Can an artistic idea be preserved through these changes of media? Can a non-trivial pattern be envisaged in an artwork, and then still be identified after the change of medium? Here, we consider a contemporary installation and an ensemble musical piece derived from…
Touch or touchless?:Evaluating usability of interactive displays for persons with autistic spectrum disorders
2019
Interactive public displays have been exploited and studied for engaging interaction in several previous studies. In this context, applications have been focused on supporting learning or entertainment activities, specifically designed for people with special needs. This includes, for example, those with Autism Spectrum Disorders (ASD). In this paper, we present a comparison study aimed at understanding the difference in terms of usability, effectiveness, and enjoyment perceived by users with ASD between two interaction modalities usually supported by interactive displays: touch-based and touchless gestural interaction. We present the outcomes of a within-subject setup involving 8 ASD users…
Facilitating joint attention with salient pointing in interactions involving children with autism spectrum disorder
2016
Children with autism spectrum disorder (ASD) reportedly have difficulties in responding to bids for joint attention, notably in following pointing gestures. Previous studies have predominantly built on structured observation measures and predefined coding categories to measure children’s responsiveness to gestures. However, how these gestures are designed and what detailed interactional work they can accomplish have received less attention. In this paper, we use a multimodal approach to conversation analysis (CA) to investigate how educators design their use of pointing in interactions involving school-aged children with ASD or autistic features. The analysis shows that pointing had specifi…
Diagrammatic Gestures. Cognition, Mathematics, and Semiotics.
2021
here are words having an extraordinary evocative power. This is precisely the case of “Diagram” and “Gesture”. The frst term covers an interconnected network of concepts like “graph”, “schema”, “form”, “model”, “arrow”, and so on... And the same can be said of the second one: just think, for example, of the ideas of “movement”, “bodily action”, “practical activity” or, more generally, of “doing”. Accounting for them has been, over the last 50 years, the aim of signifcant portions of the work conduction in philosophy (Deleuze, Foucault, Merleau-Ponty), mathematics (Grothendieck, Lawvere, Thom), semiotics (Peirce), and theoretical linguistics (Culioli, Langacker, Pottier). The present issue a…
Design and Implementation of Deep Learning Based Contactless Authentication System Using Hand Gestures
2021
Hand gestures based sign language digits have several contactless applications. Applications include communication for impaired people, such as elderly and disabled people, health-care applications, automotive user interfaces, and security and surveillance. This work presents the design and implementation of a complete end-to-end deep learning based edge computing system that can verify a user contactlessly using &lsquo